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Ensemble  Techniques  for  Determining  Globally  Optimal  Designs  for 
Problems  with  Broadly  Stated  Design  Objectives 


1 .  Purpose  of  the  Project 

Problems  in  design  of  advanced  photonic  and  electronic  devices  often  require  the  search  over  high¬ 
dimensional  design  spaces  to  optimize  highly  non-convex  performance  functions.  The  large  number  of  design 
parameters  is  also  making  it  extremely  difficult  to  rely  on  intuition  of  design  engineers  to  develop  near-optimal 
candidate  designs.  As  a  result,  local  optimization  techniques  are  not  sufficient  for  these  problems.  Prior  to  the 
start  of  this  project,  the  main  investigators  have  experimented  with  other  commonly  used  global  optimization 
approaches  including  Genetic  Algorithm  (GA)  and  Simulated  Annealing  (SA).  However,  these  techniques  tend 
to  not  make  use  of  all  available  local  information  such  as  the  gradient  and  approximated  Hessian  of  the 
performance  function.  On  the  other  hand,  the  existing  techniques  rely  on  analogies  with  natural  evolution  or 
random  motion  of  energize  particle  to  bias  the  exploration  of  the  design  space.  Unfortunately,  these  analogies 
are  often  non-intuitive  for  optimal  design  problems.  The  primary  purpose  of  this  project  is  to  make  use  of  the 
basic  ideas  of  ensemble  technique  to  develop  an  alternative  global  optimization  technique  for  global  optimal 
design  problems.  The  ensemble  technique  has  been  successfully  used  in  weather  forecasting  and  large  scale 
data  assimilation.  The  key  idea  of  ensemble  technique  is  to  use  parallel  evaluations  of  a  highly  complex 
system  model  over  a  variety  of  conditions  to  obtain  valuable  information  on  the  system.  The  new  algorithm  is 
designed  to  solve  optimization  problems  for  which  the  evaluation  of  the  performance  functional  is  very 
computationally  intensive  and  the  design  space  is  a  compact  subset  of  a  high-dimensional  vector  space.  Since 
for  many  design  problems  it  is  also  interesting  to  identify  locally  optimal  but  globally  sub-optimal  design 
solutions,  the  new  algorithm  needs  to  efficiently  distribute  ensemble  members  to  provide  as  complete  coverage 
of  the  entire  design  space  as  possible  and  identify  locally  optimal  designs  in  the  exploration  process. 

2.  Major  Achievement  of  the  Project 

In  the  short  period  of  the  performance  of  this  project,  we  have  developed  and  implemented  an  Ensemble 
Global  Optimization  (EnGO)  technique  that  allow  hierarchical  random  search  of  the  design  space  with 
progressively  finer  coverage  and,  at  the  same  time,  effectively  use  available  local  gradient  information  to 
accelerate  convergence  to  locally  and  globally  optimal  solutions.  The  exploration  of  the  design  space  through 
an  ensemble  of  candidate  designs  enables  the  algorithm  to  quickly  abandon  a  non-promising  candidate  when 
the  relative  performance  of  the  candidate  design  is  inferior  to  other  candidates.  Multiple  ensemble  statistical 
criteria  are  used  to  eliminate  redundant  search  of  the  same  area  of  the  design  space  and  to  minimize 
exploration  around  non  promising  regions  of  the  design  space. 

In  addition  to  the  development  of  the  EnGO,  we  have  developed  a  systematic  method  to  evaluate  the 
performance  of  a  global  optimization  method  on  a  class  of  randomly  generated  functions  in  high  dimensional 
vector  spaces.  This  methodology  provides  us  with  a  test-bed  to  evaluate  and  improve  EnGO.  The  class  of  the 
randomly  generated  functions  has  the  form 

j(x)  =  ]Tak  •  exp(-  (x  -  sk  f  A;1  (x  -  sk )) . 

k= 1 

The  quantities  ak ,  Ak  and  sk  are  randomly  generated  values,  covariant  matrices  and  vectors,  respectively. 

For  each  value  of  k,  the  component  of  the  function  reaches  either  maximum  or  minimum  at  the  seed  point  sk . 

The  function  trends  to  zeros  at  infinity.  The  selection  of  this  class  of  random  functions  meets  the  following 
criteria  important  for  device  design  problems: 

•  The  performance  function  is  locally  smooth  function  of  the  design  variables  and  the  gradient  of 
performance  function  can  be  efficiently  evaluate; 

•  There  is  a  small  neighborhood  around  the  globally  and  locally  optimal  design  such  that  when  a  gradient 
based  local  optimization  algorithm  starts  with  an  initial  solution  inside  of  this  neighborhood,  the 
algorithm  converges  toward  the  optimal  design,  and 

•  The  function  is  computationally  easy  to  evaluate  to  be  used  for  testing  of  global  optimization  scheme. 


An  example  of  a  random  function  in  2-dimensional  space  is  given  shown  in  Figure  1. 
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Figure  1. 2-dimensional  randomly  generated  function  (a)  and  its  gradient  field  (b) 


We  have  evaluated  EnGO  against  GA  and  SA  using  this  class  of  functions.  An  example  of  the  result  in  5D  is 
shown  in  Figure  2  and  3. 
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Figure  2.  Performance  of  EnGO  for  5D 


Figure  3.  Performance  of  GA 

Since  both  EnGO  and  GA  use  random  generation  of  candidate  designs  in  the  search  process,  it  is  important  to 
evaluate  these  algorithms  in  statistically  meaningful  way.  In  our  evaluation,  each  of  the  algorithms  is  evaluated 
over  30  repeated  test  runs  on  the  same  underlying  function.  In  each  test  run,  20  ensemble  members  are  used 
and  35  iterations  are  performed.  In  Figure  2  and  3,  the  statistics  of  the  30  minimal  values  among  the  20 


ensemble  members  for  each  of  the  test  runs  are  displayed.  After  35  generations,  a  substantial  portion  of  the 
test  runs  for  GA  is  trapped  at  a  local  minimum  in  the  design  space  while  the  EnGO  algorithm  is  successful  in 
all  test  runs  to  find  the  global  minimum  of  the  problem. 

An  important  challenge  in  high-dimensional  global  optimization  is  the  “curse  of  dimensionality”.  That  is  the 
volume  of  space  to  explore  increases  exponentially.  We  have  developed  strategy  to  work  in  high-dimensional 
problems.  In  fact,  we  have  successfully  tested  our  approach  for  a  10-dimensional  problem  as  shown  in  Figure 
4. 
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Figure  4.  Test  results  for  EnGO  on  a  10-dimensional  problem 


Future  Research 

Our  immediate  next  step  is  to  implement  the  EnGO  for  two  prototype  design  problems  in  electromagnetic 
device  design  and  electronic  device  design.  This  will  allow  us  to  demonstrate  the  potential  of  the  EnGO  in 
contributing  to  the  improvement  over  local  optimization  based  design  approaches. 

Parallel  to  the  application  of  EnGO  on  the  prototype  problems,  we  are  continuing  the  development  of  EnGO  in 
better  handling  of  high-dimensional  design  problem  and  multiple  objective  design  problems. 


